A simulation study on two analytical techniques for alternating treatments designs

Alternating treatments designs (ATDs) are single-case experimental designs entailing the rapid alternation of conditions, and the specific sequence of conditions is usually determined at random. The visual analysis of ATD data entails comparing the data paths formed by connecting the measurements from the same condition. Apart from visual analyses, there are at least two quantitative analytical options also comparing data paths. On option is a visual structured criterion (VSC) regarding the number of comparisons for which one conditions has to be superior to the other to consider that the difference is not only due to random fluctuations. Another option, denoted as ALIV (a comparison involving Actual and Linearly Interpolated Values), computes the mean difference between the data paths and uses a randomization test to obtain a p value. In the current study, these two options are compared, along with a binomial test, in the context of simulated data, representing ATDs with a maximum of two consecutive administrations of the same condition and a randomized block design. Both VSC and ALIV control Type I error rates, although these are closer to the nominal 5% for ALIV. In contrast, the binomial test is excessively liberal. In terms of statistical power, ALIV plus a randomization test is superior to VSC. We recommend that applied researchers complement visual analysis with the quantification of the mean difference, as per ALIV, and with a p value whenever the alternation sequence was determined at random. We have extended an already existing website providing the graphical representation and the numerical results

Reporting single-case design studies: Advice in relation to the designs' methodological and analytical peculiarities

The current text provides advice on the content of an article reporting a single-case design research. The advice is drawn from several sources, such as the Single-case research in behavioral sciences reporting guidelines, developed by an international panel of experts, scholarly articles on reporting, methodological quality scales, and the author's professional experience. The indications provided on the Introduction, Discussion, and Abstract are very general and applicable to many instances of applied psychological research across domains. In contrast, more space is dedicated to the Method and Results sections, on the basis of the peculiarities of single-case designs methodology and the complications in term s of data analysis. Specifically, regarding the Method, several aspects strengthening (or allowing the assessment of) the internal validity are underlined, as well as information relevant for evaluating the possibility to generalize the results. Regarding the Results, the focus is put on justifying the analytical approach followed. The author considers that, even if a study does not meet methodological quality standards, it should include sufficiently explicit reporting that makes possible assessing its methodological quality. The importance of reporting all data gathered, including unexpected and undesired results, is also highlighted. Finally, a checklist is provided as a summary of the reporting tips

Linear trend in single-case visual and quantitative analyses

The frequently used visual analysis of single-case data focuses on data aspects such as level, trend, variability, overlap, immediacy of effect, and consistency of data patterns; most of these aspects are also commonly quantified besides inspecting them visually. The present text focuses on trend, because even linear trend can be operatively defined in several different ways, while there are also different approaches for controlling for baseline trend. We recommend using a quantitative criterion for choosing a trend line fitting technique and comparing baseline and intervention slopes, instead of detrending. We implement our proposal in a free web-based application created specifically for following the What Works Clearinghouse Standards recommendations for visual analysis. This application is especially destined to applied researchers and provides graphical representation of the data, visual aids, and quantifications of the difference between phases in terms of level, trend, and overlap, as well as two quantifications of the immediate effect. An evaluation of the consistency of effects across replications of the AB sequence is also provided. For methodologists and statisticians, we include formulas and examples of the different straight line fitting and detrending techniques to improve the reproducibility of results and simulations

How can single-case data be analyzed? Software resources, tutorial, and reflections on analysis

The present article aims to present a series of software developments in the quantitative analysis of data obtained via single-case experimental designs (SCEDs), as well as the tutorial describing these developments. The tutorial focuses on software implementations based on freely available platforms such as R and aims to bring statistical advances closer to applied researchers and help them become autonomous agents in the data analysis stage of a study. The range of analyses dealt with in the tutorial is illustrated on a typical single-case dataset, relying heavily on graphical data representations. We illustrate how visual and quantitative analyses can be used jointly, giving complementary information and helping the researcher decide whether there is an intervention effect, how large it is, and whether it is practically significant. To help applied researchers in the use of the analyses, we have organized the data in the different ways required by the different analytical procedures and made these data available online. We also provide Internet links to all free software available, as well as all the main references to the analytical techniques. Finally, we suggest that appropriate and informative data analysis is likely to be a step forward in documenting and communicating results and also for increasing the scientific credibility of SCEDs

Analyzing data from single-case alternating treatments designs

Alternating treatments designs (ATDs) have received comparatively less attention than other single-case experimental designs in terms of data analysis, as most analytical proposals and illustrations have been made in the context of designs including phases with several consecutive measurements in the same condition. One of the specific features of ATDs is the rapid (and usually randomly determined) alternation of conditions, which requires adapting the analytical techniques. First, we review the methodologically desirable features of ATDs, as well as the characteristics of the published single-case research using an ATD, which are relevant for data analysis. Second, we review several existing options for ATD data analysis. Third, we propose 2 new procedures, suggested as alternatives improving some of the limitations of extant analytical techniques. Fourth, we illustrate the application of existing techniques and the new proposals in order to discuss their differences and similarities. We advocate for the use of the new proposals in ATDs, because they entail meaningful comparisons between the conditions without assumptions about the design or the data pattern. We provide R code for all computations and for the graphical representation of the comparisons involved. (PsycINFO Database Record

Recommendations for choosing single-case data analytical techniques

The current paper responds to the need to provide guidance to applied single-case researchers regarding the possibilities of data analysis. The amount of available single-case data analytical techniques has been growing during recent years and a general overview, comparing the possibilities of these techniques, is missing. Such an overview is provided that refers to techniques that yield results in terms of a raw or standardized difference and procedures related to regression analysis, as well as nonoverlap and percentage change indices. The comparison is provided in terms of the type of quantification provided, data features taken into account, conditions in which the techniques are appropriate, possibilities for meta-analysis, and evidence available on their performance. Moreover, we provide a set of recommendations for choosing appropriate analysis techniques, pointing at specific situations (aims, types of data, researchers' resources) and the data analytical techniques that are most appropriate in these situations. The recommendations are contextualized using a variety of published single-case data sets in order to illustrate a range of realistic situations that researchers have faced and may face in their investigations

Violin Plots as Visual Tools in the Meta-Analysis of Single-Case Experimental Designs

Despite the existence of sophisticated statistical methods, systematic reviews regularly indicate that single-case experimental designs (SCEDs) are predominantly analyzed through visual tools. For the quantitative aggregation of results, different meta-analytical techniques are available, but specific visual tools for the meta-analysis of SCEDs are lacking. The present article therefore describes the use of violin plots as visual tools to represent the raw data. We first describe the underlying rationale of violin plots and their main characteristics. We then show how the violin plots can complement the statistics obtained in a quantitative meta-analysis. The main advantages of violin plots as visual tools in meta-analysis are (a) that they preserve information about the raw data from each study, (b) that they have the ability to visually represent data from different designs in one graph, and (c) that they enable the comparison of score distributions from different experimental phases from different studies

Defining and assessing immediacy in single-case experimental designs

Immediacy is one of six data aspects (alongside level, trend, variability, overlap, and consistency) that has to be accounted for when visually analyzing single-case data. Given that it is one of the aspects that has received considerably less attention than other data aspects, the current text offers a review of the proposed conceptual definitions of immediacy (i.e., what it refers to) and also of the suggested operational definitions (i.e., how exactly is it assessed and/or quantified). Provided that a variety of conceptual and operational definitions is identified, we propose following a sensitivity analysis using a randomization test for assessing immediate effects in single-case experimental designs, by identifying when changes were most clear. In such a sensitivity analysis, the immediate effects are tested for multiple possible intervention points and for different possible operational definitions. Robust immediate effects can be detected if the results for the different operational definitions converge

Assessing Consistency in Single-Case Data Features Using Modified Brinley Plots

The current text deals with the assessment of consistency of data features from experimentally similar phases and consistency of effects in single-case experimental designs. Although consistency is frequently mentioned as a critical feature, few quantifications have been proposed so far: namely, under the acronyms CONDAP (consistency of data patterns in similar phases) and CONEFF (consistency of effects). Whereas CONDAP allows assessing the consistency of data patterns, the proposals made here focus on the consistency of data features such as level, trend, and variability, as represented by summary measures (mean, ordinary least squares slope, and standard deviation, respectively). The assessment of consistency of effect is also made in terms of these three data features, while also including the study of the consistency of an immediate effect (if expected). The summary measures are represented as points on a modified Brinley plot and their similarity is assessed via quantifications of distance. Both absolute and relative measures of consistency are proposed: the former expressed in the same measurement units as the outcome variable and the latter as a percentage. Illustrations with real data sets (multiple baseline, ABAB, and alternating treatments designs) show the wide applicability of the proposals. We developed a user-friendly website to offer both the graphical representations and the quantifications

Randomization tests for ABAB designs: Comparing-data-division-specific and common distributions

Monte Carlo simulations were used to generate data for ABAB designs of different lengths. The points of change in phase are randomly determined before gathering behaviour measurements, which allows the use of a randomization test as an analytic technique. Data simulation and analysis can be based either on data-division-specific or on common distributions. Following one method or another affects the results obtained after the randomization test has been applied. Therefore, the goal of the study was to examine these effects in more detail. The discrepancies in these approaches are obvious when data with zero treatment effect are considered and such approaches have implications for statistical power studies. Data-division-specific distributions provide more detailed information about the performance of the statistical technique